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I.  INTRODUCTION 


A.  BACKGROUND 

Technology  transition  has  become  an  area  of  extensive  focus  within  the 
Department  of  Defense  (DoD)  over  the  last  several  years. 


(DoD)  is  in  the  process  of  transforming  the  U.S.  armed  forces  from  a  Cold 
War-era  fighting  force  to  one  that  is  lighter,  more  flexible,  and  more 
reliant  on  technology.  This  fighting  force  will  be  able  to  respond  to  a  wide 
range  of  asymmetric  threats  with  speed  and  efficiency.  Accelerating  the 
transition  of  new  technologies  into  defense  systems  will  be  crucial  to 
achieving  this  military  transfonnation.  However,  the  typical  time  required 
for  moving  new  materials  and  processing  technologies  from  research  to 
applications  is  at  least  10  years,  and  many  times  even  longer.  Historical 
precedents  for  the  transition  of  new  technologies  into  defense  systems 
have  been  neither  fast  nor  efficient,  (www.nap.edu,  p.  1) 

To  understand  the  motivation  behind  a  shift  in  the  manner  DoD  transitions 
technologies,  one  may  simply  examine  the  statistics  of  major  systems  that  have  faltered 
during  development  due  to  the  inability  to  successfully  accelerate  the  incorporation  of 
advanced  technology. 

Faster  incorporation  of  new  technologies  into  complex  products  and 
systems  holds  the  possibility  of  ever-increasing  advantages  in  cost, 
performance,  durability,  and  new  functionalities,  (www.nap.edu,  p.  vii) 

More  efficient  and  effective  management  of  technology  is  critical  to  ensuring  the 
Warfighter  receives  timely,  innovative  solutions  within  cost  and  schedule  constraints 
while  maintaining  mission  effectiveness.  A  Memorandum  from  the  Under  Secretary  of 
Defense  (Acquisition,  Technology,  and  Logistics)  (USD(AT&L))  to  the  Secretary  of 
Defense,  Subject:  Top  5  Priorities  for  AT&L,  August  5,  2002,  clearly  stated, 

Accelerating  the  flow  of  technology  to  the  Warfighter  is  one  of  the  top 
priorities  of  the  [USD(AT&L)j,  as  well  as  the  Services,  defense  agencies, 
and  other  key  defense  organizations  that  help  transition  technology.  (DoD- 
DAU,  p.  xv) 

As  a  result,  new  organizational  structures,  policies,  and  metrics  have  evolved. 


1 


The  Valley  of  Death  occurs  around  the  Milestone  B  decision  at  the  time  the 
system  is  progressing  from  the  Technology  Development  (TD)  phase  into  the 
Engineering  &  Manufacturing  (EMD)  phase  (as  seen  in  Figure  1). 


Technology  Opportunities  &  Resources 


JCIDS  - > 


Acquisition  Process - > 


Figure  1.  Defense  Acquisition  Management  System  (From:  Matthews,  p.  40) 


Milestone  B  is  the  critical  point  in  the  Acquisition  Management  when  the 
technology  enters  the  Systems  Acquisition  Phase.  At  that  time, 

The  project  shall  exit  the  Technology  Development  Phase  when  an 
affordable  program  or  increment  of  militarily  useful  capability  has  been 
identified;  the  technology  and  manufacturing  processes  for  that  program 
or  increment  have  been  assessed  and  demonstrated  in  a  relevant 
environment;  manufacturing  risks  have  been  identified;  a  system  or 
increment  can  be  developed  for  production  within  a  short  timeframe 
(normally  less  than  5  years  for  weapon  systems);  or,  when  the  MDA 
decides  to  terminate  the  effort.  During  Technology  Development,  the  user 
shall  prepare  the  Capability  Development  Document  (CDD)  to  support 
initiation  of  the  acquisition  program  or  evolutionary  increment,  refine  the 
integrated  architecture,  and  clarify  how  the  program  will  lead  to  joint 
warfighting  capability.  The  CDD  builds  on  the  ICD  and  provides  the 
detailed  operational  performance  parameters  necessary  to  complete  design 
of  the  proposed  system.  A  Milestone  B  decision  follows  the  completion  of 
Technology  Development.  (AT&F,  p.  19) 
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The  period  of  time  between  TD  and  EMD  also  correlates  with  a  Technology 
Readiness  Level  (TRL)  6,  as  defined  by  the  Defense  Acquisition  University  and  shown  in 
Figure  2,  as  “System/subsystem  model  or  prototype  demonstration  in  a  relevant 
environment.”  Historically,  the  S&T  community  has  developed  technologies  through 
TRL  5,  yet,  acquisition  programs  focus  on  more  mature  technologies  at  the  TRL  7. 


FRP 

MS 

C 

MS 

B 


MS 

A 


System  Test,  Flight 
and  Operations 

9  Actual  system  “flight  proven"  through 
successful  mission  operations 

System/Subsystem 

Development 

8  Actual  system  completed  and  “flight  qualified 
through  test  and  demonstration 

7  System  prototype  demonstration  in  an 
operational  environment 

Technology  Demonstration 

6  System/subsystem  model  or  prototype 
demonstration  in  a  relevant  environment 

Technology  Development 

5  Component  and/or  breadboard  validation  in 
relevant  environment 

4  Component  and/or  breadboard  validation  in 
laboratory  environment 

Research  to  Prove  Feasibility 

3  Analytical  and  experimental  critical  function 
and/or  characteristic  proof-of-concept 

Basic  Technology  Research 

2  Technology  concept  and/or  application 
formulated 

1  Basic  principles  observed  and  reported 

Figure  2.  Acquisition  Cycle  and  Technology  Readiness  Levels 


B.  DEFINITION 

A  vital  step  in  the  authors’  research  was  to  define  technology  transition.  Contrary 
to  the  common  wisdom,  technology  transition  and  technology  transfer  are  significantly 
different.  The  Defense  Acquisition  University  defines  technology  transition  in  the 
following  manner: 

The  process  of  applying  critical  technology  in  military  systems  to  provide 
an  effective  weapons  and  support  system — in  the  quantity  and  quality 
needed  by  the  Warfighter  to  carry  out  assigned  missions  and  at  the  “best 
value”  as  measured  by  the  Warfighter. 
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The  authors  suggest  that  the  concept  of  time  be  included  in  the  definition.  In 
today’s  geopolitical  instability  and  in  the  face  of  relentless  adversaries,  it  is  essential  that 
the  DoD  performs  technology  transition  in  a  practical  yet  expedient  manner.  Time  is  a 
critical  factor  in  technology  transition. 

On  the  other  hand. 

Technology  Transfer  is  the  transfer  or  licensing  of  technology  developed 
in  Federal  Laboratories  to  domestic  partners,  including  industry,  state  and 
local  governments,  and  academia.  Key  mechanism  for  the  transfer  are 
Cooperative  Research  and  Development  Agreements  (CRADAs),  Patent 
License  Agreements,  Educational  Partnership  Agreements,  and  DoD-wide 
Partnership  Intennediaries.  (acc.dau.mil,  n.p.) 

Technology  transfer,  although  a  critical  component  of  technology  development, 
shall  not  be  discussed  in  this  document. 

C.  CHALLENGES 

All  programs  and  initiatives  will  face  expected  and  unexpected  challenges. 
Historical  evidence  shows  DoD’s  difficulties  overcoming  these  challenges  in 
transitioning  technologies  from  S&T  to  EMD. 

DOD  relies  on  its  research  laboratories  and  test  facilities  as  well  as 
industry  and  academia  to  develop  new  technologies  and  systems  that 
improve  and  enhance  military  operations  and  ensure  technological 
superiority  over  adversaries.  Yet,  historically,  DOD  has  experienced 
problems  in  bringing  technologies  out  of  the  lab  environment  and  into  real 
use.  At  times,  technologies  do  not  leave  the  lab  because  their  potential  has 
not  been  adequately  demonstrated  or  recognized.  In  other  cases, 
acquisition  programs — which  receive  the  bulk  of  DOD’s  funding  in 
research,  development,  testing  and  evaluation  of  technology — are  simply 
unwilling  to  fund  final  stages  of  development  of  a  promising  technology, 
preferring  to  invest  in  other  aspects  of  the  program  that  are  viewed  as 
more  vital  to  success.  Other  times,  they  choose  to  develop  the 
technologies  themselves,  rather  than  rely  on  DOD  labs  to  do  so — a 
practice  that  brings  cost  and  schedule  risk  since  programs  may  well  find 
themselves  addressing  problems  related  to  technology  immaturity  that 
hamper  other  aspects  of  the  acquisition  process.  And  often,  DOD’s 
budgeting  process,  which  requires  investments  to  be  targeted  at  least  2 
years  in  advance  of  their  activation,  makes  it  difficult  for  DOD  to  seize 
opportunities  to  introduce  technological  advances  into  acquisition 
programs.  In  addition,  it  is  challenging  just  to  identify  and  pursue 
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technologies  that  could  be  used  to  enhance  military  operations  given  the 
very  wide  range  of  organizations  inside  and  outside  of  DOD  that  are 
focused  on  technology  development  and  the  wide  range  of  capabilities  that 
DOD  is  interested  in  advancing.  (GAO,  p.  4) 

Along  with  the  aforementioned  challenges,  three  other  contributory  factors  have 
constrained  technology  transition. 

•  Lack  of  policy  guiding  and/or  enforcing  the  implementation  of  technology 
transition  processes 

•  Lack  of  organizational  initiatives  and  structure  that  encourage  and  foster  a  strong 
technology  transition  environment 

•  Lack  of  a  standardized  metrics-based  system  to  measure  the  maturity  of  a 
technology  and  the  ability  to  correlate  that  information  with  the  acquisition 
framework 

Addressing  the  challenges  with  a  systematic  approach  has  allowed  for  a  gradual 
shift  in  mindset  within  DoD;  the  changes  are  being  embraced. 

D.  NOTIONAL  CONSTRUCT 

Transitioning  technology,  or  ’’bridging  the  gap”  as  organizations  such  as  DARPA 
refer  to  it,  is  a  complicated  task  requiring  a  tremendous  cooperative  effort  amongst  those 
shown  in  Figure  3.  The  combination  of  organizational  transformation,  policy 
adjustments,  and  a  metric-based  recognition  of  technology  maturation  provides  a  DoD  a 
path  forward  to  cross  the  dreaded  valley  of  death. 
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Figure  3.  Collaborative  Influences 


As  indicated  in  Figure  3,  there  is  a  close-knit  interdependence  between  the 
Warfighter,  the  Requirements  Office,  the  Industry  Partner,  and  the  Stakeholder,  which 
influences  the  resultant  end-item  solution.  This  tight  interaction  is  critical  throughout  the 
technology  transition  phase  to  ensure  the  appropriate  solution  is  delivered  to  the 
Warfighter.  Unfortunately,  this  arrangement  has  historically  delayed  the  rapid  insertion 
of  technology  due  predominantly  to  the  requirements  portion.  DoD  programs  are  highly 
requirements-driven  programs;  the  requirements  take  lengthy  periods  of  time  to  derive. 
Therefore,  DoD  made  a  strategic  decision  per  the  2001  Quadrennial  Defense  Review  in 
which  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  was  stood  up 
to  replace  DoD’s  legacy  Requirements  Generation  System  (GAO,  p.  4) 
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The  JCIDS  process  requires  that  gaps  in  military  capabilities  be  identified 
and  potential  materiel  and  nonmateriel  solutions  for  filling  those  gaps  be 
developed  based  on  formal  capability  assessments.  The  results  of  these 
capability  assessments  are  formally  submitted  as  initial  capabilities 
documents  (ICD) — a  capability  proposal — by  a  military  service,  defense 
agency,  COCOM,  FCB,  or  other  sponsor.  ICDs  are  intended  to  document 
a  specific  capability  gap  or  set  of  gaps  that  exist  in  joint  warfighting 
functions  and  propose  a  prioritized  list  of  various  solutions  to  address  the 
gap(s).  (GAO,  p.  5) 

The  GAO  study  continues  to  state: 

The  JCIDS  process  may  lack  the  efficiency  and  agility  needed  to  respond 
to  warfighter  needs — especially  those  that  are  near  term — because  the 
review  and  validation  of  capability  proposals  can  take  a  significant  amount 
of  time.  A  proposal  submitted  to  JCIDS  can  go  through  several  review  and 
comment  resolution  phases  before  consensus  is  reached  on  the  proposal, 
and  through  several  levels  of  approval  before  the  JROC  [Joint 
Requirements  Oversight  Council]  validates  the  proposal.  Our  review  of 
capability  proposals  submitted  to  JCIDS  from  fiscal  years  2003  through 
2008  found  that  review  and  validation  takes  on  average  8  to  10  months 
(see  fig.  5).  JCIDS  and  service  officials  also  indicated  that  prior  to 
submitting  a  JCIDS  proposal,  the  sponsor  can  take  a  year  or  more  to 
complete  a  capabilities-based  assessment  and  get  a  proposal  approved.  In 
other  words,  2  years  or  more  can  elapse  from  the  time  a  capability  need  is 
identified  by  a  sponsor  to  the  time  the  capability  is  validated  by  the  JROC. 

(GAO,  p.  13) 

What  if  a  requirement  does  not  exist,  but  a  technology  or  prototype  developed  in 
the  S&T  community  has  the  potential  to  be  of  great  benefit  to  the  Warfighter? 
Unfortunately,  in  most  if  not  all  cases  this  transition  will  not  occur  due  to  the  inability  to 
acquire  funding.  This  lack  of  appropriated  funding  to  support  non-JROC  sanctioned 
efforts  is  disruptive  to  the  technology  development  process  and  detrimental  to  mission 
effectiveness.  An  organizational  shift  to  include  funding  set-asides  and  a  bubble-up  or 
technology  push  approach  would  more  effectively  address  the  immediate  needs  of  the 
Warfighter.  The  men  and  women  engaged  in  the  daily  activities  of  the  battlespace  can 
offer  a  tremendous  amount  of  insight  into  what  is  needed.  As  opposed  to  waiting  for 
vetted  requirements  to  drive  technology  development,  allow  concepts  and  prototypes  to 
bubble-up  through  the  ranks,  consequently  utilizing  the  buy-in  of  the  operator  community 
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to  address  a  capability  gap  with  a  specific  technology.  This  improved  organizational 
process  would  increase  the  probability  of  meeting  or  exceeding  the  perfonnance 
expectations  of  the  Warfighter  and  reduce  the  time  necessary  to  hand-off  the  end-item 
solution. 

E.  HISTORICAL  BACKGROUND 

The  attention  given  to  technology  transition  is  a  fairly  new  initiative  as  compared 
to  the  lengthy  history  of  DoD.  Official  legislation  introducing  the  Technology  Transition 
Initiative  (TTI)  was  established  by  Congress  in  the  FY2003  National  Defense 
Authorization  Act.  TTI,  enacted  by  the  107th  Congress  of  the  United  States  of  America, 
was  detailed  as  follows  in  H.R.4546: 

(a)  Initiative  Required  -  The  Secretary  of  Defense,  acting  through  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics, 
shall  carry  out  an  initiative,  to  be  known  as  the  Technology  Transition 
Initiative  (hereinafter  in  this  section  referred  to  as  the  “Initiative”),  to 
facilitate  the  rapid  transition  of  new  technologies  from  science  and 
technology  programs  of  the  Department  of  Defense  into  acquisition 
programs  of  the  Department  for  the  production  of  such  technologies. 


(b)  Objectives  -  The  objectives  of  the  Initiative  are  as  follows: 


(1)  To  accelerate  the  introduction  of  new  technologies  into  operational 
capabilities  for  the  anned  forces. 

(2)  To  successfully  demonstrate  new  technologies  in  relevant 
environment.  (107th  Congress,  p.  37) 

Prior  to  the  enactment  of  H.R.4546,  limited  activity  occurred  within  DoD  to 
improve  upon  the  past  performance  of  technology  transition.  One  example  of  such  an 
initiative  is  the  Advanced  Concept  and  Technology  Demonstration  (ACTD)  Program. 

The  Department  of  Defense's  traditional  approach  to  developing  and 
building  weapon  and  information  systems  has  been  criticized  for  taking 
too  long,  costing  too  much,  and  not  adequately  involving  those  who 
ultimately  use  the  equipment.  To  address  those  problems,  the  Department 
of  Defense  (DoD)  initiated  the  Advanced  Concept  Technology 
Demonstration  (ACTD)  program  in  1994.  (Joseph,  n.p.) 
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The  ACTD  process,  as  seen  in  Figure  4,  addresses  the  technology  maturity  and 
insertion  issues  abating  technology  transition  from  the  TD  Phase  to  the  EMD  Phase. 
However,  process  alone  was  not  enough  to  ensure  a  successful  transition. 


AC  TD  Prcies 


TriiWinml 
A(  qiifition 
Ercceff 


Figure  4.  ACTD  Process  (From:  Joseph,  n.p.). 


When  DoD  established  the  ACTD  program  in  1994,  it  did  not  include  a 
formal  approach  to  address  any  concerns  that  might  arise  about  the 
transition  stage  of  the  projects.... For  ACTDs  that  began  in  and  after  1996, 
the  AT  Office  (Office  of  the  Deputy  Under  Secretary  of  Defense  for 
Advanced  Technology)  made  one  important  change  by  requiring  project 
managers  to  establish  transition  integrated  product  teams  (IPTs)  where 
applicable.  Transition  IPTs  examine  whether  a  program  will  be  affordable, 
what  the  requirements  for  operating  and  supporting  the  residual  capability 
will  be,  what  strategies  are  needed  for  testing  and  evaluation,  and  whether 
there  is  adequate  documentation  for  maintenance,  training,  or  moving  a 
system  into  procurement.  (Joseph,  n.p.) 

The  ACTD  Program  has  gradually  transitioned  to  the  Joint  Concept  and 
Technology  Demonstration  Program  or  JCTD;  it  “was  first  introduced  into  the 
President’s  Budget  in  2006”  (Peterson,  p.  9). 
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The  new  Joint  Capability  Technology  Demonstration  Program  (JCTD) 
furthers  this  [the  ACTD]  concept  by  developing  and  maturing 
technologies  to  support  the  unique  needs  of  the  joint  community  in  an 
even  more  adaptive  and  responsive  process.  (Krieg,  p.  7) 

The  JCTD  program  falls  within  the  structure  of  the  Rapid  Fielding  Directorate, 
OSD.  The  Rapid  Fielding  mission  is,  “Rapidly  transition  innovative  concepts  into  critical 
capabilities  that  counter  unconventional  and  time-sensitive  threats” 
(www.acq.osd.mil/rfd/,  n.p.). 

Since  the  inception  of  the  ACTD/JCTD  process,  several  additional  initiatives  and 
organizations  have  emerged.  (The  authors  address  these  in  Chapter  III.)  Yet  one 
prominent  organization  evolved  well  before  ACTDs  and  other  technology  transition 
initiatives — the  Defense  Advanced  Research  Projects  Agency  (DARPA). 

DARPA  was  created  in  1958  as  the  Advanced  Research  Projects  Agency 
(ARP A).  The  political  and  defense  communities  recognized  the  need  for  a 
high-level  defense  organization  to  formulate  and  execute  R&D  projects 
that  would  expand  the  frontiers  of  technology  beyond  the  immediate  and 
specific  requirements  of  the  Military  Services  and  their  laboratories. 
(www.darpa.mil.  History) 

Since  DARPA’ s  inception,  the  organization  has  proven  to  be  very  successful 
maturing  technologies  and  establishing  technology  transition  strategies.  A  study 
conducted  by  the  Potomac  Institute  for  Policy  Studies  in  2001  presented  the  following 
regarding  the  success  of  DARPA’s  technology  transition  initiatives. 

The  principal  finding  of  the  study  is  that  DARPA's  transition  performance 
has  been  excellent  over  the  past  forty  years,  inserting  over  120  products  or 
technologies  into  fielded  systems  (about  3  per  year).  During  the  past 
decade,  the  Agency's  record  has  been  even  better,  about  5  per  year. 

Finally,  where  data  was  available,  we  calculated  transition  rates  and  found 
them  to  be  at  a  level  exceptionally  high  according  to  industry's  standards. 
Considering  DARPA's  other  missions  and  its  responsibility  to  foster  high- 
risk/high-payoff  ideas,  the  Institute's  team  considers  these  statistics  quite 
impressive.  (Richardson,  n.p.) 
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II.  TECHNOLOGY  TRANSITION  POLICY 


A.  INTRODUCTION 

This  section  discusses  the  technology  transition  policies  within  the  DoD.  The 
major  polices  governing  technology  transition  within  the  DoD  are  found  in  DoD 
Directive  5000.01,  DoD  Instruction  5000.02,  and  in  the  Defense  Acquisition  Guidebook. 
The  major  policies  related  to  technology  transition  are  explored  below. 

B.  DOD  DIRECTIVE  5000.01 

DoD  Directive  5000.01  outlines  the  Defense  Acquisition  System.  It  is  applicable 
to  all  acquisition  programs  within  DoD.  This  Directive  defines  the  Defense  Acquisition 
System  (DAS)  as  “the  management  process  by  which  the  Department  of  Defense 
provides  effective,  affordable,  and  timely  systems  to  the  users”  (p.  4).  It  defines  an 
Acquisition  Program  as  a  “directed,  funded  effort  that  provides  a  new,  improved,  or 
continuing  materiel,  weapon  or  information  system,  or  service  capability  in  response  to 
an  approved  need”  (p.  4). 

The  purpose  of  the  DAS  is  to: 

Manage  the  nation's  investments  in  technologies,  programs,  and  product 
support  necessary  to  achieve  the  National  Security  Strategy  and  support 
the  United  States  Armed  Forces.  The  investment  strategy  of  the 
Department  of  Defense  shall  be  postured  to  support  not  only  today's  force, 
but  also  the  next  force,  and  future  forces  beyond  that.  (DoD  Directive 
5000.01,  p.  3) 

It  further  states  that 

the  primary  objective  of  Defense  acquisition  is  to  acquire  quality  products 
that  satisfy  user  needs  with  measurable  improvements  to  mission 
capability  and  operational  support,  in  a  timely  manner,  and  at  a  fair  and 
reasonable  price.  (DoD  Directive  5000.01,  p.  3) 

In  order  to  ensure  the  continued  technological  superiority  of  U.S.  forces  over  our 
adversaries’  successful  technology  transition  is  a  key  component  in  the  DAS.  DoD 
Directive  5000.01  specifically  addresses  technology  development  and  transition. 
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It  requires  that  a  program: 

Address  user  needs;  maintain  a  broad-based  program  spanning  all 
Defense-relevant  sciences  and  technologies  to  anticipate  future  needs  and 
those  not  being  pursued  by  civil  or  commercial  communities;  preserve 
long-range  research;  and  enable  rapid,  successful  transition  from  the  S&T 
base  to  useful  military  products.  (DoD  Directive  5000.01,  pp.  9-10) 

C.  DOD  INSTRUCTION  5000.02 

DoD  Instruction  5000.02  implements  DoD  Directive  5000.01  and  lays  out  the 
operation  of  the  DAS.  It  is  applicable  to  all  acquisition  programs  within  DoD.  The 
Defense  Acquisition  Management  System  prescribed  in  the  DoD  Instruction  5000.02  is 
illustrated  in  Figure  5. 


Technology  Opportunities  &  Resources 


•  The  Materiel  Development  Decision  precedes 
entry  into  any  phase  of  the  acquisition 
management  system 

•  Entrance  criteria  met  before  entering  phase 

•  Evolutionary  Acquisition  or  Single  Step  to 
Full  Capability 
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Figure  5.  Acquisition  Cycle 


Concerning  technology  transition,  the  technology  development  phase  of  this 
management  system  is  of  key  importance.  During  this  phase,  promising  new 
technologies  must  be  matured  to  a  sufficient  state  allowing  them  to  bridge  the  Valley  of 
Death  occurring  around  the  Milestone  B  decision  and  be  successfully  transitioned  into  an 
acquisition  program. 

The  purpose  the  technology  development  phase  is  to: 

Reduce  technology  risk,  determine  and  mature  the  appropriate  set  of 
technologies  to  be  integrated  into  a  full  system,  and  to  demonstrate  CTEs 
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[Critical  Technology  Elements]  on  prototypes.  Technology  Development 
is  a  continuous  technology  discovery  and  development  process  reflecting 
close  collaboration  between  the  S&T  community,  the  user,  and  the  system 
developer.  It  is  an  iterative  process  designed  to  assess  the  viability  of 
technologies  while  simultaneously  refining  user  requirements.  (DoD 
Instruction  5000.02,  p.  16) 

In  order  to  enter  the  technology  development  phase  the  Milestone  Decision  Authority 
(MDA)  must  approve  the  material  solution  and  the  Technology  Development  Strategy 
(TDS).  The  TDS  must  explain  the  proposed  acquisition  approach,  either  single  step  to 
full  capability,  or  evolutionary,  which  is  the  preferred  approach.  When  an  evolutionary 
strategy  is  being  used,  the  TDS  must  explain  the  increments  that  the  material  solution 
will  be  divided  into  given  the  maturity  of  the  technology  involved  in  the  system. 

DoD  Instruction  5000.02  further  states: 

If  an  evolutionary  strategy  is  used,  the  initial  capability  represents  only 
partial  fulfillment  of  the  overall  capability  described  in  the  ICD,  and 
successive  technology  development  efforts  continue  until  all  capabilities 
have  been  achieved.  In  an  evolutionary  acquisition,  the  identification  and 
development  of  the  technologies  necessary  for  follow-on  increments 
continue  in  parallel  with  the  acquisition  of  preceding  increments,  allowing 
the  mature  technologies  to  more  rapidly  proceed  into  the  Engineering  and 
Manufacturing  Development  (EMD)  Phase.  Each  increment  of  an 
evolutionary  acquisition  program  that  includes  a  Milestone  A  shall  have  a 
Milestone  Decision  Authority  (MDA)-approved  TDS.  (p.  18) 

DoDI  5000.02  requires  that  technology  risk  be  managed  and  mitigated  in  order  to 
help  ensure  the  program  meets  its  cost  and  schedule  goals.  To  this  end,  DoDI  5000.02 
requires  routine  objective  assessments  of  technology  maturity  and  risk.  DoDI  5000.02 
states  that 

Technology  developed  in  S&T  or  procured  from  industry  or  other  sources 
shall  have  been  demonstrated  in  a  relevant  environment  or,  preferably,  in 
an  operational  environment  to  be  considered  mature  enough  to  use  for 
product  development,  (p.  19). 

This  corresponds  to  at  least  TRL  6.  DoDI  5000.02  requires  that  Technology  Readiness 
Assessments  be  conducted  and  if  the  technology  is  detennined  not  to  meet  the  minimum 
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maturity  level  then  an  alternative  technology  meeting  the  minimum  maturity  level  and 
satisfying  the  users’  requirements  must  be  used.  If  that  is  not  possible,  then  modifying  the 
users’  requirements  is  discussed  with  the  user. 

The  DoD  Technology  Readiness  Assessment  (TRA)  Deskbook  (2009)  defines  a 

TRA  as 

a  formal,  systematic,  metrics-based  process  and  accompanying  report  that 
assesses  the  maturity  of  technologies  called  Critical  Technology  Elements 
(CTEs)  to  be  used  in  systems,  (p.  1-1). 

CTEs  are  defined  as 

A  technology  element  is  “critical”  if  the  system  being  acquired  depends  on 
this  technology  element  to  meet  operational  requirements  (within 
acceptable  cost  and  schedule  limits)  and  if  the  technology  element  or  its 
application  is  either  new  or  novel  or  in  an  area  that  poses  major 
technological  risk  during  detailed  design  or  demonstration.  (DoD,  2009,  p. 

1-1) 

In  order  to  help  clarify  this  definition,  the  DoD  TRA  Deskbook  further  states  that 
“If  the  technology  represents  a  major  risk,  it  should  be  identified  as  a  CTE  so  that  the 
TRA  will  include  technical  information  that  can  be  used  to  mitigate  the  risk”  (p.  1-1). 
TRAs  are  conducted  by  an  independent  review  team  (IRT)  composed  of  the  necessary 
subject  matter  experts  (SMEs).  DoD  acquisition  programs  must  prepare  a  TRA  at 
Milestone  B  and  C.  In  accordance  with  10  USC  2366b  on  Major  Defense  Acquisition 
Programs  (MDAPs)  the  MDA  must  certify  that  the  CTEs  are  at  least  at  TRL  6.  The  law 
allows  the  MDA  to  waive  this  certification;  however,  this  is  only  in  extraordinary 
circumstances. 

In  summary,  the  DoDI  5000.02  requires  that  CTEs  be  at  least  at  TRL  6  in  order  to 
be  used  in  the  system  past  Milestone  B.  So,  before  and  during  the  technology 
demonstration  phase  technology  elements  identified  as  critical  must  be  managed  so  that 
they  are  matured  to  at  least  TRL  6  or  else  they  will  not  be  usable  in  that  increment  of  the 
program.  These  technologies  could  be  slated  for  further  maturation  and  potential 
insertion  into  a  future  increment  of  the  program  once  they  have  reached  TRL  6  or  they 
could  be  shelved  with  no  plan  for  future  use. 
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D.  DEFENSE  ACQUISITION  GUIDEBOOK 


The  Defense  Acquisition  Guidebook  (DAG)  is  meant  to  complement  DoDD 
5000.01  and  DoDI  5000.02  and  provides  additional  guidance  to  the  Acquisition 
Community  on  implementing  these  policies.  Relative  to  technology  transition  the  DAG 
provides  additional  information  concerning  the  content  of  the  TDS.  The  DAG  specifies 
that  the  TDS  should  include  a  technology  maturation  section.  It  states, 

The  Technology  Maturation  section  of  the  TDS  should  include  discussion 
of  the  identified  preliminary  critical  technology  elements  (CTEs)  and  the 
respective  strategies  for  attaining  at  least  Technology  Readiness  Level 
(TRL)  6  prior  to  Milestone  B.  (p.  2.2.8) 

The  goals  for  the  maturation  of  each  of  the  CTEs  should  be  laid  out  including  any 
reviews,  tests,  and/or  demonstrations  of  how  the  CTEs  are  maturing.  As  previously 
discussed  for  CTEs,  a  TRL  level  of  at  least  6  must  be  reached  prior  to  Milestone  B.  A 
TRL  6  indicates  the  CTE  has  been  demonstrated  in  a  relevant  environment.  Therefore,  in 
order  to  make  this  assessment,  the  relevant  environment  must  be  defined  in  the  context  of 
the  specific  system  being  developed. 

The  Defense  Acquisition  Guidebook  states  that  the: 

Technology  Development  Strategy  should  identify  the  relevant 
environment  including  potential  threats  against  the  capabilities  desired  and 
the  physical  environment  (altitude,  temperature,  vibration,  electro¬ 
magnetic,  etc...),  within  which  the  materiel  solution  needs  to  operate.  The 
characteristics  of  the  environment  should  also  be  related  to  testing  efforts 
in  the  Test  and  Evaluation  Strategy,  (p.  2.2.8. 1) 

DoDI  5000.02  requires  competitive  prototyping  involving  at  least  two  competing 
teams  prior  to  Milestone  B  approval.  The  DAG  provides  additional  guidance  to  DoD 
acquisition  professionals  regarding  competitive  prototyping.  The  DAG  states  that  the 
TDS  “should  include  a  description  of  the  prototyping  purpose  and  the  prototyping 
strategy  at  the  system  and  subsystem  levels”  (p.  2. 2. 8. 2). 
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It  further  states, 

The  prototyping  strategy  should  be  competitive  and  provide  for  prototypes 
of  the  system  or,  if  a  system  prototype  is  not  feasible,  for  prototypes  of 
critical  subsystems  before  Milestone  (MS)  B  approval,  (p.  2. 2. 8. 2) 

The  use  of  competitive  prototyping  at  the  system  or  subsystem  level  is  a  valuable  tool  to 
help  mature  technology,  reduce  risk,  provide  the  government  with  a  better  understanding 
of  the  technology,  and  increase  contractor  motivation  (DoD,  2010). 

DoDI  5000.02  requires  Program  Managers  (PMs)  to  consider  the  use  of  Small 
Business  Research  Innovation  (SBIR)  program  technologies  and  “...give  favorable 
consideration  to  successful  SBIR  technologies”  (p.  14).  The  DAG  provides  further 
implementing  guidance  stating  that  the  PM  should 

...prepare  a  Technology  Development  Strategy  (TDS)  that  appropriately 
uses  the  SBIR  program  to  develop  needed  technologies,  includes  the  use 
of  technologies  developed  under  the  SBIR  program,  and  gives  fair 
consideration  to  successful  SBIR  technologies,  (p.  2.2.10.1) 

The  PM  must  examine  available  SBIR  technologies  for  inclusion  in  the  system, 
either  in  the  current  increment  or  for  possible  inclusion  in  a  subsequent  increment  and 
plan  accordingly  to  develop  those  technologies  to  the  appropriate  maturation  level  and 
transition  them  into  the  program. 
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III.  ORGANIZATION 


A.  PROGRAMS 

In  the  post-Cold  War  era,  a  change  was  necessary  to  remain  the  most 
technologically  superior  military  in  the  world.  At  that  point,  DoD  began  to  implement 
organizational  transformations  to  account  for  a  leaner,  rapid  response  military  with  a 
global  perspective.  The  DoD  Acquisition  System,  up  to  the  early  ‘90s,  was  prohibitive 
to  rapid,  cooperative  technology  development  and  insertion.  In  order  to  address  this 
predicament,  DoD  established  several  organizations  that  utilized  non-traditional,  industry 
practices  to  abate  the  difficulties  bringing  technologies  to  a  suitable  maturation  state. 
Albeit  these  organizations  may  not  remedy  the  technology  transition  difficulties,  their 
intent  was  a  paradigm  shift  in  DoD  acquisition.  The  following  chart  describes  a  portion 
of  the  DoD  organizational  initiatives. 


Program 

Management 

Characteristics  that 
Facilitate  Transition 

Advanced  Concept/Joint 
Capabilities  Technology 
Demonstration  (AC/JCTD) 

Program  direction  and 
oversight  by  OSD 

Projects  managed  and 
executed  by  Military 
Departments,  Defense 
Agencies  and  SOCOM 

Focuses  on  Joint  Warfighter 
Functional  Capabilities 
which  are  often  under¬ 
resourced  and  with  no  one 
organization  tasked  to  meet 
these  needs 

Allows  warfighters  to 
evaluate  a  technology  for  its 
potential 

Agile  Integration  and 
Development 

Army 

Funds  acceleration  for 
selected  high-payoff 
emerging 
technologies 

Improves  technology 
readiness  by  accelerating 
the  tech  development 
schedule  and/or  performing 
detailed  safety  &  validation 
tests  in  field/operational 
environment 
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Collaborative 

Technology  Alliances 

Army 

Collaboration  among 
Government -Industry- 
University  researchers  to 
achieve  affordable  transition 
of  innovative  technologies 

Defense  Acquisition 
Challenge  Program 
(DAC) 

OSD  oversight 

Military  Department/ 
Defense  Agency 
management  and  execution 

Provides  an  “on-ramp”  for 
industry  and  government  to 
propose  innovative  new 
technology  and  equipment 
solutions  for  acquisition 
programs 

Foreign  Comparative 

Testing  (FCT) 

OSD  oversight 

Military  Department/ 
Defense  Agency 
management  and  execution 

Helps  find  developed 
technologies  in  allied 
nations  and  funds  their 
testing  for  potential 
procurement 

Future  Naval 

Capabilities  (FNC) 

Navy 

Involves  near-term  S&T 
efforts  that  deliver  maturing 
technologies  for  more 
timely  incorporation 

Rapid  Deployment 

Capability  (RDC) 

Navy 

Provides  the  ability  to  reach 
immediately  to  a  newly 
discovered  threat  or  to 
respond  to  significant  and 
urgent  safety 
situations 

Rapid  Technology 

Transition  Program 
(RTT) 

Navy 

Has  a  charter  to  rapidly 
transition  technology  from 
any  source,  including  those 
not  traditionally  associated 
with  defense  technology, 
into  PORs 

SBIR  Commercialization 
Pilot  Program  (CPP) 

Army,  Navy,  Air  Force 

OSD  Oversight,  Direction, 
and  Coordination 

Identifies  projects  with 
potential  to  meet  high 
priority  military  needs 

Provides  assistance  to  affect 
rapid  transition 

Technology  Insertion  for 
Savings  Program  (TIPS) 

Navy 

Concentrates  on  existing 
and/or  COTS  technology 
that  can  be  fonned  quickly 
into  the  solution  and  has  a 
high  return  on  investment 
after  insertion 
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Technology  Transition 
Initiative  (TTI) 

Program  direction  and 
oversight  by  OSD 

Projects  managed  and 
executed  by  Military 
Departments,  Defense 
Agencies  and  SOCOM 

Accelerate  the  introduction 
of  new  technologies  into 
operational  capabilities  for 
the  anned  forces 

Warfighter  Rapid 

Acquisition  Process 

Air  Force 

Provides  transition  funding 
for  development  &  fielding 
of  successful  experiments, 
demonstrations,  & 
innovative  approaches 

Complete,  approved 
acquisition  plan 

Budget  Activity  7 

Table  1.  Technology  Transition  Organizational  Initiatives 


(Source:  Department  of  Defense  Report  to  Congress  on  Technology  Transition,  p.  12) 


All  of  these  organizations  foster  a  strong,  open-communicative  architecture, 
beginning  with  the  user  (the  Warfighter)  progressing  through  the  S&T  and  EMD  arenas. 
They  were  formed  to  fulfill  a  common  goal — rapidly  transition  mature  technologies  to 
the  Warfighter.  Whether  the  organization  is  supplying  the  funding  to  bridge  the  S&T  and 
EMD  appropriation  gap  or  providing  over-arching  management  or  actually  developing 
and  maturing  the  technology,  they  have  impacted  the  Warfighter  with  highly  functional, 
relevant  solutions.  Yet,  much  larger  strides  in  organizational  reform  are  needed  to 
overcome  DoD’s  historically  slow  development  cycle.  Even  after  Mr.  David  Packer  of 
the  Blue  Ribbon  Commission  on  Defense  Management  lamented  over  the  “unreasonably 
long  acquisition  life  cycle  of  10-15  years”  (Packer,  p.  47)  in  1986  DoD  still  lacks  the 
ability  to  reduce  the  acquisition  cycle  of  a  major  system.  This  unfortunately 

leads  to  obsolete  technology  in  our  fielded  equipment  [and]  we  forfeit  our 
five-year  technological  lead  by  the  time  it  takes  us  to  get  our  technology 
from  the  laboratory  into  the  field.  Packer,  p.  47) 
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In  addition  to  addressing  a  broad  technology  base,  the  various  organizations  and 
initiatives  listed  in  Table  1  span  across  the  joint  services,  a  critical  attribute  in  the 
modern-day  military  environment  where  a  large  portion  of  programs  are  jointly 
represented.  This  joint  approach  is  intended  to  leverage  the  knowledge,  skills,  and 
abilities  across  the  services  and  maintain  commonality  to  more  rapidly  mature  a 
technology.  Solutions  may  have  their  slight  differences  between  the  services;  however, 
they  have  a  common  goal  to  transition  useful  technologies  such  as  the  F-35  Joint  Strike 
Fighter  (JSF)  Program.  The  JSF  Program,  originally  referred  to  as  the  Joint  Advanced 
Strike  Technology  (JAST)  Program,  was  a  joint  effort  between  the  Navy,  Air  Force, 
Marines,  and  the  UK. 

The  goal  of  the  JAST  program  was  not  to  have  developed  a  new  aircraft, 
but  instead  it  was  to  mature  the  technologies  that  a  new  series  of  tactical 
aircraft  could  use.  (www.jsf.mil,  History/JAST) 


Tech  Link 


DACP 

9% 


D PA  Title  I 
4% 


FC- 

110, 


JCTD 

12% 


ACTD 

51% 


Figure  6.  Technology  Transition  Initiative  Funding  Break-out 

(Source:  Payton,  Slide  9) 
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Program 

$M 

Advanced  Concept 
Technology 
Demonstration  (ACTD) 
(BA-3) 

$164 

Joint  Capability 
Technology 
Demonstration  (JCTD) 
(BA-3/4/5  &  Procure) 

$40 

Joint  Warfighting 
Program  (JWP)-BA-3 

$10 

Foreign  Comparative 
Testing  (FCT}-BA-6 

$36 

Defense  Production 

Act (DPA)  Title  III 

$13 

Tech  nology  T  ra  nsition 
Initiative  (TTI)-BA-3 

$30 

TechLink  (BA-3) 

$3 

Defense  Acquisition 
Challenge  Pgm 
(DACP)-BA-5 

$29 

Table  2.  Technology  Transition  Initiative  Funding  ($) 
(Source:  Payton,  Slide  9) 


As  demonstrated  in  Figure  1  and  Table  2,  a  sufficient  amount  of  funding  has  been 
injected  into  these  new  programs.  Based  upon  these  amounts  and  the  number  of  new 
initiatives,  it  is  clear  the  need  for  a  more  efficient  technology  transition  process  has  been 
recognized  within  DoD.  DoD  must  maintain  vigilant  oversight  to  ensure  efforts  that  all 
within  these  and  forthcoming  programs  do  not  become  victims  of  traditional  10-15  year 
programs. 

B.  COOPERATIVE  INITIATIVES 

Refonn  of  the  acquisition  process  is  now  a  driving  force  in  the  Department 
of  Defense.  A  number  of  specific  acquisition  reform  initiatives  have  been 
conceived,  some  borrowed  from  industry,  and  briefed  at  the  highest  levels 
of  Government.  Many  have  been  mandated  by  the  Secretary  of  Defense, 

Dr.  William  Perry,  implemented  by  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology,  Dr.  Paul  Kaminski,  the  Services,  and  some 

21 


have  been  enacted  by  Congress.  One  such  initiative  borrowed  from 
industry  that  will  fundamentally  change  the  way  the  Department  does 
business  is  Integrated  Product  and  Process  Development  (IPPD).  (Burt, 
Foreword) 

The  IPPD  process,  initiated  in  the  early  1990s 

is  the  DoD  management  technique  that  simultaneously  integrates  all 
essential  acquisition  activities  through  the  use  of  multidisciplinary  teams 
to  optimize  design,  manufacturing,  and  supportability  processes.  One  of 
the  key  IPPD  tenets  is  multidisciplinary  teamwork  through  Integrated 
Product  Teams  (IPTs).  (acc.dau.mil) 

IPTs  have  become  a  necessity  in  the  development  of  technologies,  and  are  an 
essential  mechanism  to  ensure  the  right  technology  gets  to  the  right  person  at  the  right 
time.  Subject  matter  experts  from  all  of  the  acquisition  disciplines  are  brought  together 
in  a  cohesive  team  to  develop  a  technology  from  concept  through  sustainment.  This  is 
contrary  to  the  traditional  stovepipe  development  processes  historically  followed  within 
DoD.  IPTs  are  extremely  user-oriented  in  how  they  approach  development.  Users  are 
empowered  to  work  directly  with  scientists,  engineers,  manufacturing  specialists, 
logisticians,  etc.  to  “shape”  a  solution  that  most  appropriately  balances  the  requirements 
with  the  “wants”  and  “like  to  haves.”  This  constant,  open  communication  and  interaction 
is  of  great  benefit  as  a  technology  matures  and  is  eventually  inserted  into  a  system 
solution.  It  has  been  demonstrated  on  several  large  acquisition  programs  that  well- 
managed  IPTs  are  extremely  successful  transitioning  technologies  through  the  TRLs  and 
MRLs  and  their  associated  milestones. 

C.  INDUSTRY  PROCEDURES 

The  business  practices  of  industry  are  a  great  asset  to  DoD  and  provide  valuable 
lessons  learned.  Although  industry  does  not  follow  the  stringent  acquisition  regulations 
that  DoD  is  required  to  adhere  to  and  their  predominant  focus  is  profit,  the  manner  by 
which  they  have  transitioned  technology  has  proven  to  be  very  successful.  According  to 
a  GAO  study,  industry  leaders  focus  on  three  key  techniques: 
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•  Strategic  planning  at  the  corporate  level 

•  A  gated  management  review  process  that  ensures  a  technology’s 
relevancy,  feasibility,  and  transition  readiness 

•  Effective  tools  to  solidify  commitment,  address  transition  issues, 
and  gauge  project  progress  and  process  effectiveness.  (GAO,  p.  9) 

To  expand  upon  these  techniques: 


Successful  transition  in  leading  companies  starts  with  strong  strategic 
planning  followed  by  a  structured  technology  development  process  led  by 
research  labs  and  supported  by  tools  that  pave  the  way  for  a  smooth 
handoff  to  the  product  line.  Strategic  planning  is  considered  a  precursor  to 
transition  and  allows  managers  to  identify  market  needs  so  the  company 
can  quickly  adapt  its  technology  portfolios  to  meet  those  needs.  A  gated 
technology  development  process  continually  tests  for  relevancy  and 
feasibility  of  technologies  and  gauges  the  commitment  of  product  line 
managers  to  accept  them.  Once  a  technology  is  ready  to  transition, 
management  and  funding  responsibilities  gradually  shift  from  the  lab  to 
the  product  line.  By  the  end  of  transition,  but  before  product  development 
starts,  the  technology  is  validated  as  mature  enough  for  use  in  the  intended 
product.  (GAO,  p.  3) 


TECHNOLOGY  DEVELOPMENT  GATES 

Explore 

Technology  Ideas  and  concepts 
are  being  explored 

ffev 

Develop 

Technology  development  activities 
are  underway 

lew  flev 

k  J 

Technology  transition 

Technology  is  ready  to  transition 
from  lab  to  product  line  team 

few 

k 

Deliverables 

♦  Technology  is  consistent  with  overall 
business  strategy 

♦  Technology  is  promising  and  is  likely  to 
meet  needs  for  potential  product  lines 

♦  Lab  identifies  potential  products  where 
technology  can  be  used 

♦  Key  cost,  benefit,  risk,  marketing, 
manufacturing,  and  life  cycle  manage¬ 
ment  issues  are  identified 

♦  Scalability  approaches  are  identified 
.  Technologies  considered  to  be 

intellectual  property  are  identified 

Deliverables 

•  Technology  is  consistent  wilh  techno¬ 
logy  strategy  and  other  relevant 
strategies 

•  Labs  have  high  degree  of  confidence 
the  technology  will  work 

•  Product  line  team  agrees  that  the 
technology  will  meet  its  needs 

•  Technical  requirements  are  identified 

•  Cost,  benefits,  and  risks  are  quantified 

•  Scalability  approach  is  selected 

•  Strategies  for  addressing  intellectual 
property  rights  are  selected 

Deliverables 

•  Technology  protect  complies  with 
technology  strategy 

.  Technology  is  sound 

•  Technology  meets  product  require¬ 
ments 

•  Cost,  benefit,  and  risks  are  well 
understood 

•  Technology  can  be  scaled  to  a 
magnitude  appropriate  for  practical 
application 

•  Product  line  team  agrees  technology  is 
ready 

•  Intellectual  property  rights  methods 
have  been  pursued 

•  Technology  is  demonstrated  in  an 
operational  environment 

•  Technical  documentation  is  ready  to  be 
given  to  product  line  team 

Figure  7.  Technology  Development  Gates  (From:  GAO,  p.  12) 


23 


Depicted  in  Figure  7,  each  phase  of  an  industry  development  cycle  is  gated  and 
successful  transition  into  the  subsequent  phase  is  predicated  upon  the  completion  of  the 
deliverables.  This  strategy  builds  upon  the  concept  of  TRLs  and  MRLs  as  metrics 
throughout  the  acquisition  life  cycle.  Both  provide  the  PM  with  objective  criteria  to 
determine  the  maturity  of  a  technology. 

Relationship  managers  are  another  key  industry  technique  that  has  rarely  been 
implemented  within  DoD.  These  managers  are  a  critical  communication  focal  point 
throughout  the  entire  product  development  life  cycle. 

With  the  exception  of  DARPA,  DOD  does  not  use  relationship  managers 
in  the  same  manner  as  leading  private  companies.  According  to  an  Air 
Force  lab  official,  relationship  managers  market  technologies  being 
developed  by  the  labs  or  gather  data  about  ongoing  projects  for  senior  lab 
management.  Most  communication  about  technology  transition  in  DOD 
takes  place  through  integrated  product  teams  or  during  annual  reviews  of 
technology  projects  by  the  senior-level  oversight  boards  for  each  of  the 
services.  Use  of  relationship  managers  for  these  purposes  are  helpful,  but 
the  managers  do  not  necessarily  serve  as  points  of  contact  within  the  labs 
and  acquisition  communities,  do  not  devote  time  toward  efficiently 
transitioning  technologies  to  multiple  weapon  system  programs,  and  do 
not  help  identify  and  address  systemic  transition  problems. 

Within  DARPA,  senior  officers,  called  operational  liaisons,  focus  on 
marketing  and  transitioning  DARPA-sponsored  technologies.  According 
to  the  DARPA  director  [Dr.  Anthony  Tether],  the  liaisons  have  been  very 
helpful  with  transitioning  technologies  because  they  are  well  practiced  at 
using  the  command  chain  of  their  respective  services  and  finding  the  right 
service  contact  at  the  right  time.  The  liaisons: 

•  Provide  operational  advice  for  planning  and  strategy  development 

•  Provide  an  understanding  of  service  perspectives,  issues  and  needs 
so  that  potential  customers  can  be  identified  and  effective 
agreements  can  be  written 

•  Draft  and  coordinate  agreements  between  DARPA  and  the  services 

•  Direct  technology  insertion  in  the  services.  (GAO,  pp.  32-33) 
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As  a  direct  link  to  the  Case  Study  in  Chapter  V, 


The  DARPA  [prior]  director  [Dr.  Anthony  Tether]  credits  operational 
liaisons  for  the  quick  transition  of  the  Boomerang,  an  acoustic  shot- 
detection  system,  from  the  lab  to  troops  in  Iraq.  DARPA  developed  the 
system  in  response  to  feedback  from  Iraq  that  convoys  were  being 
engaged  by  snipers  yet  remained  unaware  of  sniper  attacks  until  a 
windshield  was  broken,  a  soldier  was  hit,  or  a  vehicle  was  visibly 
damaged  upon  inspection  at  the  end  of  the  convoy  mission.  Within  60 
days  of  an  urgent  Army  request,  DARPA  fielded  the  first  Boomerang 
system.  But  DARPA’s  director  said  the  system  did  not  hold  up  well  in  the 
extreme  weather  conditions  and  under  wartime  conditions.  As  Boomerang 
II  was  being  prepared  for  fielding,  the  director  said  the  operational  liaisons 
helped  craft  a  more  realistic  concept  of  operations,  training  package,  and 
logistical  support  package  to  ensure  that  Boomerang  II  not  only  was 
technologically  ready  for  combat  but  was  properly  supported  with  spare 
parts,  maintenance  facilities,  maintenance  personnel,  training,  and  lessons- 
leamed  feedback  to  DARPA  and  the  Army.  The  liaisons  also  ensured  that 
the  Army  acquisition  community  was  alerted  to  Boomerang  II’s 
deployment  so  product  developers  would  be  ready  to  evaluate  the  final 
product  for  movement  into  the  more  traditional  acquisition  process. 
(GAO,  p.  33) 


An  organizational  transformation  is  slowly  infiltrating  DoD,  yet  traditional 
roadblocks  are  inhibiting  a  more  rapid  progression.  DoD’s  lack  of  success  is  made 
evident  in  Table  3: 
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Table  1:  Cost  and  Schedule  Outcomes  tor  23  Programs  Initiated  under  the  Revised  Policy  (as  of  December  2005) 

Program 

Percent  growth  in  estimated 
development  cost* 

Percent  growth  in  estimated 
development  schedule 

Expeditionary  Fighting  Vehicle 

61% 

70% 

Active  Electronically  Scanned  Array  radar  (upgrade 
for  F/A-t  8  E'F  fighter/attack  aircraft) 

14% 

1% 

Global  Hawk  unmanned  aerial  vehicle 

166% 

Undetermined 

Joint  Strike  Fighter 

30% 

23% 

UH-60M  helicopter  upgrade 

151% 

25% 

C-i  30  Avionics  Modernization  Program 

122% 

Undetermined 

C-5  Reliability  Enhancement  and  Re-enginmg 

Program 

0% 

25% 

Joint  Tactical  Radio  System  Cluster  i 

31% 

44% 

Joint  Tactical  Radio  System  Waveform 

44% 

Undetermined 

Advanced  Anti-radiation  Guided  Missile 

7% 

0% 

Multi-Platform  Radar  Technology  Insertion  Program 

0% 

Undetermined 

Future  Combat  System 

48% 

53% 

E-2  Advanced  Hawkeye 

5% 

0% 

Warfighter  Information  Network-Tactical 

0% 

0% 

Small  Diameter  Bomb 

0% 

0% 

EA-18G 

7% 

0% 

Joint  Tactical  Radio  System  Cluster  5 

0% 

2% 

Multi-Mission  Maritime  Aircraft 

0% 

0% 

Standard  Missile-6  Extended  Range  Active  Missile 

Block  t 

0% 

0% 

Aenal  Common  Sensor 

45% 

36% 

B-2  Radar  Modernization  Program 

0% 

0% 

Patrioti'Medium  Extended  Air  Defense  System 

Combined  Aggregate  Program  (fire  unit) 

0% 

0% 

Mission  Planning  System 

0% 

0% 

Table  3.  DoD  Major  Programs  Cost  and  Schedule  Performance 

(Source:  GAO,  p.  36) 


The  topics  addressed  above  are  a  select  few  of  the  contributing  factors  to  the 
future  success  of  DoD’s  technology  transition  abilities.  Despite  the  numerous  programs, 
initiatives,  and  lessons  learned  from  industry,  DoD’s  traditional  methods  of  conducting 
business  are  still  prevalent  throughout  the  acquisition  cycle,  in  particular,  the  technology 
transition  phase.  Unfortunately  until  sound  processes,  procedures,  and  techniques  are 
fully  implemented  and  adhered  to  the  Warfighter  suffers  the  greatest  impact.  It  is 
incumbent  upon  DoD  to  not  only  promote  an  organizational  transformation  with  regards 
to  technology  transition  but  to  take  a  step  back  and  look  at  the  big  picture;  absorb  the 
positive  attributes  of  the  current  structure  and  discard  the  negative  attributes.  The 
organizational  mindset  alone  can  be  a  powerful  tool  to  bridge  the  gap  between  S&T  and 
Engineering. 
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IV.  TECHNOLOGY  METRICS 


A.  BACKGROUND 

As  stated  in  the  DoD’s  report  to  Congress  in  July  2007,  “Transitioning  technology 
into  established  Programs  of  Record  (PoR)  is  a  longstanding  challenge  for  the 
Department.  The  underlying  problem  has  come  into  sharp  focus  in  recent  years  with  the 
adoption  of  Technology  Readiness  Levels  (TRLs)  as  a  common  vocabulary  for 
discussing  the  maturity  of  a  technology  and  uncovering  the  disconnect  between  our  S&T 
and  acquisition  communities.  Our  acquisition  policies  require  a  minimum  of  TRL  7 
("system  prototype  demonstrated  in  an  operational  environment")  for  a  critical 
technology  to  be  incorporated  in  a  production  program  (an  important  best  practice 
recommended  by  GAO  to  control  technical  risk,  and  strongly  supported  by  DoD).  On  the 
other  hand,  expectations  within  S&T  community  have  traditionally  been  to  advance  new 
technologies  only  to  the  TRL  5  level  ("component  and/or  breadboard  validation  in  a 
relevant  environment"  (e.g.,  high  fidelity  laboratory)),  with  no  particular  capability 
deployment  in  mind,  and  then  move  on  to  the  next  technology.  Although  a  technology 
maturity  gap  exists,  one  now  has  the  ability  to  exploit  a  metrics-based  system  to  measure 
the  breadth  of  that  gap  through  Technology  Readiness  Levels  (TRLs).  Therefore,  TRLs 
are  not  simply  an  assessment  tool  but  also  a  planning  and  risk  mitigation  tool. 

The  introduction  of  Technology  Readiness  Levels  (TRLs)  over  the  last 
decade  provided  an  accepted  common  language  and  measurement  scale 
intended  to  strengthen  communication  within  and  between  the  DoD  S&T 
and  acquisition  communities,  both  in  government  and  industry. 
Stakeholders  in  all  parts  of  the  acquisition  system  now  expect  that  a 
Critical  Technology  Element  (CTE)  will  have  an  appropriate  TRL,  or 
level  of  maturity,  prior  to  acceptance  as  a  baseline  technology  for  a 
weapon  system.  They  also  embrace  the  concept  that  technology  should 
have  only  an  acceptable  level  of  risk  in  order  to  pass  through  each 
acquisition  milestone  decision  point  and  that  TRLs  are  a  way  of  measuring 
and  communicating  that  risk.  (JDMTP,  ES-5) 
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B.  TECHNOLOGY  READINESS  LEVELS 

In  1991,  the  Office  of  Aeronautics  and  Space  Technology  (OAST)  of  the  National 
Aeronautical  and  Space  Administration  (NASA)  developed  an  Integrated  Technology 
Plan  (HP). 

The  purpose  of  the  ITP  was  to  serve  as  a  strategic  plan  for  the  OAST 
space  research  and  technology  (R&T)  program  [...]  conducting 
technology  development  that  support  future  U.S.  civil  space  missions. 
(NASA,  ii) 

Within  the  ITP,  nine  definitive  levels  of  technology  maturity  were  defined.  Those 
nine  levels  (shown  in  Table  4)  have  become  the  backbone  of  measuring  technology 
development;  although  the  definitions  have  evolved  somewhat  over  the  years  (and  across 
different  organizations)  the  general  premise  remains  the  same.  Technology  development 
requires  a  metric  to  measure  the  maturity  of  the  technology  to  ensure  the  appropriate 
level  of  effectiveness  is  obtained  prior  to  transitioning  to  the  next  phase.  Until  NASA’s 


ITP,  this  uniform  means  of  assessment  did  not  exist. 


Technology  Readiness  Level 

Description 

1.  Basic  principles  observed  and  reported. 

Lowest  level  of  technology  readiness. 
Scientific  research  begins  to  be  translated 
into  applied  research  and  development. 
Examples  might  include  paper  studies  of  a 
technology’s  basic  properties. 

2.  Technology  concept  and/or  application 
formulated. 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be 
invented.  Applications  are  speculative  and 
there  may  be  no  proof  or  detailed  analysis  to 
support  the  assumptions.  Examples  are 
limited  to  analytic  studies. 

3.  Analytical  and  experimental  critical 
function  and/or  characteristic  proof  of 
concept. 

Active  research  and  development  is  initiated, 
including  analytical  studies  and  laboratory 
studies  to  physically  validate  analytical 
predictions  of  separate  elements  of  the 
technology.  Examples  include  components 
that  are  not  yet  integrated  or  representative. 

4.  Component  and/or  breadboard  validation 
in  laboratory  environment. 

Basic  technological  components  are 
integrated  to  establish  that  they  will  work 
together.  This  is  relatively  “low  fidelity” 
compared  to  the  eventual  system.  Examples 
include  integration  of  “ad  hoc”  hardware  in 
the  laboratory. 
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Technology  Readiness  Level 

Description 

5.  Component  and/or  breadboard  validation 
in  relevant  environment. 

Fidelity  of  breadboard  technology  increases 
significantly.  The  basic  technological 
components  are  integrated  with  reasonably 
realistic  supporting  elements  so  it  can  be 
tested  in  a  simulated  environment.  Examples 
include  “high  fidelity”  laboratory  integration 
of  components. 

6.  System/subsystem  model  or  prototype 
demonstration  in  a  relevant  environment. 

Representative  model  or  prototype  system, 
which  is  well  beyond  that  of  TRL  5,  is  tested 
in  a  relevant  environment.  Represents  a 
major  step  up  in  a  technology’s 
demonstrated  readiness.  Examples  include 
testing  a  prototype  in  a  high-fidelity 
laboratory  environment  or  in  simulated 
operational  environment. 

7.  System  prototype  demonstration  in  an 
operational  environment. 

Prototype  near,  or  at,  planned  operational 
system.  Represents  a  major  step  up  from 
TRL  6,  requiring  demonstration  of  an  actual 
system  prototype  in  an  operational 
environment  such  as  an  aircraft,  vehicle,  or 
space.  Examples  include  testing  the 
prototype  in  a  test  bed  aircraft. 

8.  Actual  system  completed  and  qualified 
through  test  and  demonstration. 

Technology  has  been  proven  to  work  in  its 
final  form  and  under  expected  conditions.  In 
almost  all  cases,  this  TRL  represents  the  end 
of  true  system  development.  Examples 
include  developmental  test  and  evaluation  of 
the  system  in  its  intended  weapon  system  to 
determine  if  it  meets  design  specifications. 

9.  Actual  system  proven  through  successful 
mission  operations. 

Actual  application  of  the  technology  in  its 
final  form  and  under  mission  conditions, 
such  as  those  encountered  in  operational  test 
and  evaluation.  Examples  include  using  the 
system  under  operational  mission  conditions. 

Table  4.  Technology  Readiness  Levels 
(Source:  DAU,  Ask  a  Professor) 


Clarifying  definitions  (via  DAU/Ask  a  Professor): 

•  BREADBOARD:  Integrated  components  that  provide  a 
representation  of  a  system/subsystem  and  which  can  be  used  to  determine 
concept  feasibility  and  to  develop  technical  data.  Typically  configured  for 
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laboratory  use  to  demonstrate  the  technical  principles  of  immediate 
interest.  May  resemble  final  system/subsystem  in  function  only. 


•  HIGH  FIDELITY:  Addresses  form,  fit  and  function.  High  fidelity 
laboratory  environment  would  involve  testing  with  equipment  that  can 
simulate  and  validate  all  system  specifications  within  a  laboratory  setting. 

•  LOW  FIDELITY :  A  representative  of  the  component  or  system  that 
has  limited  ability  to  provide  anything  but  first  order  infonnation  about  the 
end  product.  Low  fidelity  assessments  are  used  to  provide  trend  analysis. 

•  MODEL:  A  reduced  scale,  functional  form  of  a  system,  near  or  at 
operational  specification.  Models  will  be  sufficiently  hardened  to  allow 
demonstration  of  the  technical  and  operational  capabilities  required  of  the 
final  system. 

•  OPERATIONAL  ENVIRONMENT:  Environment  that  addresses  all 
of  the  operational  requirements  and  specifications  required  of  the  final 
system  to  include  platform/packaging. 

•  PROTOTYPE:  The  first  early  representation  of  the  system  which 
offers  the  expected  functionality  and  performance  expected  of  the  final 
implementation.  Prototypes  will  be  sufficiently  hardened  to  allow 
demonstration  of  the  technical  and  operational  capabilities  required  of  the 
final  system. 

•  RELEVANT  ENVIRONMENT:  Testing  environment  that  simulates 
the  key  aspects  of  the  operational  environment. 

•  SIMULATED  OPERATIONAL  ENVIRONMENTAL:  Environment 
that  can  simulate  all  of  the  operational  requirements  and  specifications 
required  of  the  final  system  or  a  simulated  environment  that  allows  for 
testing  of  a  virtual  prototype  to  detennine  whether  it  meets  the  operational 
requirements  and  specifications  of  the  final  system. 


Technology  Readiness  Assessments 

The  implementation  within  DoD  of  a  TRL  system  to  assess  the  maturity  of  a 
technology  has  more  vividly  exposed  the  gap  between  S&T  and  Engineering.  As 
components/systems  traverse  through  the  various  early  development  phases,  the  S&T, 
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Engineering  and  Test,  and  Stakeholder  communities  now  have  the  appropriate 
technology  metrics  to  prepare  for  the  Valley  of  Death  by  exploiting  the  usage  of 
Technology  Readiness  Assessments  (TRAs). 

Per  DoD  Instruction  5000.2,  the  TRA  is  a  regulatory  information 
requirement  for  all  acquisition  programs.  The  TRA  is  a  systematic, 
metrics-based  process  that  assesses  the  maturity  of  Critical  Technology 
Elements.  The  TRA  should  be  conducted  concurrently  with  other 
Technical  Reviews,  specifically  the  Alternative  Systems  Review,  System 
Requirements  Review,  or  the  Production  Readiness  Review.  If  a  platform 
or  system  depends  on  specific  technologies  to  meet  system  operational 
threshold  requirements  in  development,  production,  and  operation,  and  if 
the  technology  or  its  application  is  either  new  or  novel,  then  that 
technology  is  considered  a  Critical  Technology  Element. 

The  TRA  should  not  be  considered  a  risk  assessment,  but  it  should  be 
viewed  as  a  tool  for  assessing  program  risk  and  the  adequacy  of 
technology  maturation  planning.  The  TRA  scores  the  current  readiness 
level  of  selected  system  elements,  using  defined  Technology  Readiness 
Levels.  The  TRA  highlights  critical  technologies  and  other  potential 
technology  risk  areas  that  require  program  manager  attention. 
(dag.dau.mil.  Section  4. 3. 2.4) 

These  analyses,  along  with  a  conducive  organizational  mindset,  enhance  the  ability  of  the 
technology  developers  to  plan  for  maturation  to  TRL  6  and  ultimately  TRL  7. 

The  more  information  a  PM  is  able  to  obtain  about  their  technology’s 
developmental  progression  the  better  they  can  plan  and  develop  technology  transition 
agreements  and  establish  the  appropriate  technology  transition  partners. 

C.  MANUFACTURING  READINESS  LEVELS 

Another  metric  available  to  Program  Managers  are  Manufacturing  Readiness 
Levels  (MRLs). 

Manufacturing  Readiness  is  the  ability  to  harness  the  manufacturing, 
production,  quality  assurance,  and  industrial  functions  to  achieve  an 
operational  capability  that  satisfies  mission  needs — in  the  quantity  and 
quality  needed  by  the  war-fighter  to  carry  out  assigned  missions  at  the 
“best  value”  as  measured  by  the  Warfighter,  (dag.dau.mil.  Section 
4.4.14.2) 
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Throughout  the  development  process,  even  during  the  early  phases,  appropriate 
production  planning  is  critical  to  a  successful  transition.  During  TRAs  MRLs  are 
influential  factors  in  the  decision  to  transition  a  technology  from  the  S&T  environment 
into  the  EMD  phase.  Evaluators  will  ask,  “Is  the  item  producible?  “Can  the  item  be 
produced  in  a  cost  effective  manner?”  and  “Are  there  current  processes  to  produce  the 
item?”  These  questions,  and  many  others,  help  define  the  readiness  of  a  technology  and 
its  ability  to  bridge  the  gap  between  TRL  5  and  TRL  7.  Even  if  one  conceptualizes  the 
most  amazing  design,  if  you  cannot  practically  manufacture  it  then  it  serves  no  purpose  to 
the  Warfighter. 

There  are  ten  MRLs.  These  levels  directly  relate  to  the  nine  Technology 
Readiness  Levels  that  are  in  use  with  an  additional  MRL  10  that  is  equal  to  a  program  in 
full  rate  production.  The  first  three  levels  are  discussed  as  a  single  level,  which  is  equal  to 
TRLs  1  through  3.  Below  are  definitions  of  each  MRL  and  a  description  of  the  criteria 
necessary  to  each  level  and  the  associated  acquisition  phase: 


MRL  1-3 

The  organization  has  identified  manufacturing  concepts.  This  is  the  Pre- 
Concept  Refinement  phase.  Identification  of  current  manufacturing 
concepts  or  producibility  needs  has  occurred  and  is  based  on  laboratory 
studies. 

MRL  4 

The  organization  has  validated  the  system,  component,  or  item  in 
laboratory  environment.  This  is  the  lowest  level  of  production  readiness. 
The  Concept  Refinement  (CR)  phase  leads  to  a  Milestone  A  decision. 
Technologies  must  have  matured  to  at  least  TRL  4.  At  this  point  few 
requirements  have  been  validated  and  there  are  large  numbers  of 
engineering/design  changes.  The  organization  has  not  defined  component 
physical  and  functional  interfaces.  Materials,  machines  and  tooling  have 
been  demonstrated  in  a  laboratory  environment.  Inspection  and  test 
equipment  have  been  demonstrated  in  a  laboratory  environment.  DTC  & 
Production  drivers  identified.  Producibility  assessments  have  been  initiated. 

MRL  5 

The  organization  has  validated  component  or  item  in  an  initial  relevant 
environment.  Engineering  application/bread  board,  brass  board 
development  is  occur-ring.  This  is  the  first  half  of  the  Technology 
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Development  (TD)  phase  and  merges  with  the  second  half  when  we  begin 
system  validation  leading  to  a  Milestone  B  decision.  Technologies  must 
have  matured  to  at  least  TRL  5.  At  this  point  all  requirements  have  not 
been  validated  and  there  are  significant  engineering/design  changes. 
Industrial  Base  analysis  has  been  accomplished  to  identify  potential 
sources.  Initial  producibility  of  component  technology  has  been  completed. 
Form,  Fit  &  Function  constraints  identified  and  allocated  at  component 
level.  Key  Performance  Parameters  allocated  at  component  level  and  initial 
evaluation  of  Key  Characteristics  accomplished.  Subsystem  and  major 
component  level  DTC  goals  established.  Manufacturing  cost  considerations 
affect  technology  choices.  Manufacturing  cost  drivers/goals  identified. 
DTC/Production  costs  estimated  and  tracked.  Required  Manufacturing 
Technology  (ManTech)  efforts  initiated.  Yield/rate  issues  identified.  Key 
Quality  Characteristics  identified.  Science  and  Technology/  Special  Test 
Equipment  (ST/STE)  requirements  identified.  Initial  Manufacturing  Plan  is 
developed. 

MRL  6 

The  organization  has  validated  the  system  in  an  initial  relevant 
environment.  Engineering  application/bread  board,  brass  board 
development  is  occurring.  This  is  the  2nd  half  of  the  Technology 
Development  (TD)  phase  and  leads  to  a  Milestone  B  decision. 
Technologies  must  have  matured  to  at  least  TRL  6.  All  requirements  have 
not  been  validated  and  there  are  significant  engineering/design  changes. 
Component  physical  and  functional  interfaces  have  not  been  defined. 
Materials,  machines  and  tooling  have  been  demonstrated  in  a  relevant 
environment  but  most  manufacturing  processes  are  in  development  (e.g. 
ManTech  initiatives).  Inspection/test  equipment  has  been  demonstrated  in  a 
laboratory  environment.  Producibility  assessments  are  ongoing  initial 
improvements  begun.  Production  cost  drivers  and  goals  are  being  analyzed 
and  set.  DTC  goals  have  been  set. 

MRL  7 

The  System,  component  or  item  is  in  advanced  development.  This  is  the 
System  Development  &  Demonstration  Phase  (pre  DRR).  All  technologies 
have  matured  to  at  least  TRL  7.  At  this  point  engineering/design  changes 
should  be  decreasing.  Physical  and  functional  interfaces  should  be  clearly 
defined.  All  raw  materials  are  in  production  and  available  to  meet  the 
planned  LRIP  schedule.  Pilot  line  manufacturing  processes  have  been  set¬ 
up  and  are  under  test.  Processes  and  procedures  have  been  demonstrated  in 
a  production  relevant  environment.  During  this  phase,  the  producibility 
improvements  should  be  underway.  DTC  estimates  are  within  125%  of  the 
DTC  goals.  Production  estimates  are  being  established. 
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MRL  8 

The  system  is  in  System  Development  &  Demonstration  leading  to  a 
Milestone  C  decision.  Component  or  item  is  in  advanced  development  and 
ready  for  low  rate  initial  production.  Technologies  must  have  matured  to  at 
least  TRL  8.  Engineering/design  changes  should  be  decreasing 
significantly.  There  must  be  very  few  changes  at  the  end  of  this  phase. 
Physical  and  functional  interfaces  should  be  clearly  defined.  All  raw 
materials  are  in  production  and  are  available  to  meet  the  planned  LRIP 
schedule.  Manufacturing  processes  and  procedures  have  been  proven  on 
the  pilot  line,  under  control  and  ready  for  low  rate  initial  production. 
During  this  phase,  producibility  risk  assessments  should  be  completed.  The 
DTC  goals  should  have  been  met.  Production  estimates  meet  production 
goals. 

MRL  9 

The  system,  component  or  item  has  been  previously  produced  or  is  in 
production.  Or,  the  system,  component  or  item  is  in  low  rate  initial 
production.  This  phase  is  Low  Rate  Production  &  Deployment  leading  to  a 
Full  Rate  Production  Decision  (FRP).  During  low  rate  initial  production  all 
systems  engineering/design  requirements  should  be  met  and  there  should 
be  minimal  system  engineering/design  changes.  Technologies  must  have 
matured  to  at  least  TRL  9.  All  materials  are  in  production  and  available  to 
meet  planned  production  schedules.  All  manufacturing  processes  are 
established  and  controlled  in  production  to  three-sigma  or  some  other 
appropriate  quality  level.  Machines,  tooling  and  inspection  and  test 
equipment  deliver  three-sigma  or  some  other  appropriate  quality  level  in 
production.  Production  risk  monitoring  is  ongoing.  LRIP  costs  meet 
production  goals. 

MRL  10 

The  system,  component  or  item  previously  produced  or  in  production.  Or, 
the  system,  component  or  item  is  in  full  rate  production.  This  is  the  Full 
Rate  Production  or  Sustainment  phase.  This  is  the  highest  level  of 
production  readiness.  There  are  minimal  engineering/design  changes. 
System,  component  or  item  is  in  production  or  has  been  produced  and 
meets  all  engineering,  perfonnance,  quality  and  reliability  requirements. 
All  materials,  manufacturing  processes  and  procedures,  inspection  and  test 
equipment,  controlled  in  production  to  six-sigma  or  some  other  appropriate 
quality  level  in  production.  A  proven,  affordable  product  able  to  meet 
required  schedule.  Production  goals  meet  actual.  (JDMTP,  16-18) 
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During  the  era  of  stovepipe  development,  manufacturability  and  producibility 
were  afterthoughts  in  the  development  process.  As  DoD  has  learned  over  many  years, 
this  lack  of  insight  into  downstream  processes  is  a  recipe  for  disaster.  Hence,  the 
introduction  of  MRLs  and  Manufacturing  Readiness  Assessments  (MRAs)  into  the  pre- 
EMD  phases  of  the  development  cycle.  These  assessments  are  critical  elements  in 
determining  the  true  maturity  of  a  technology  and  the  potential  big  picture  impact  on  the 
system.  Similar  to  TRLs,  MRLs  provide  a  direct  correlation  to  the  acquisition  life  cycle. 
Additionally,  MRLs  are  an  excellent  tool  to  assist  in  the  transition  between  a  prototype 
and  production  units,  which  is  critical  to  attain  the  higher  TRLs. 

D.  SYSTEM  READINESS  LEVELS 

Technology  and  Manufacturing  Readiness  Levels  focus  on  the  maturity  of 
components  and  sub-component  whereas  they  do  not  fully  assess  the  interoperability  and 
interfacing  of  the  components  in  a  system.  To  encompass  the  overall  maturity  and  ability 
of  a  system  to  transition  a  broader  metric  is  warranted.  The  Stevens  Institute  of 
Technology  has  suggested  the  usage  of  System  Readiness  Levels  (SRLs),  which  combine 
TRLs  and  Integration  Readiness  Levels  (IRLs). 

The  SRL  Model  is  a  function  of  the  individual  Technology  Readiness 
Levels  (TRL)  in  a  system  and  their  subsequent  integration  points  with 
other  technologies,  the  Integration  Readiness  Level  (IRL).  (Sauser,  p.  10) 

The  IRLs  and  SRLs  are  outlined  in  Table  5. 
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IRL 

Definition  [9] 

7 

The  integration  of  technologies  has  been  verified  and  validated  with  sufficient 
detail  to  be  actionable. 

6 

The  integrating  technologies  can  accept,  translate,  and  structure  information 

for  its  intended  application. 

5 

There  is  sufficient  control  between  technologies  necessary  to  establish, 
manage,  and  terminate  the  integration. 

4 

There  is  sufficient  detail  in  the  quality  and  assurance  of  the  integration  between 
technologies. 

3 

There  is  compatibility  (i.e.  common  language)  between  technologies  to  orderly 
and  efficiently  integrate  and  interact. 

2 

There  is  some  level  of  specificity  to  characterize  the  interaction  (i.e.  ability  to 
influence)  between  technologies  through  their  interface. 

1 

An  interface  (i.e.  physical  connection)  between  technologies  has  been  identified 
with  sufficient  detail  to  allow  characterization  of  the  relationship. 

Table  5.  Integration  Readiness  Levels 
(Source:  Sauser,  p.  11) 
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SRL 

Name 

Definition 

5 

Operations  &  Support 

Execute  a  support  program  that  meets 
operational  support  performance 
requirements  and  sustains  the  system  in 
the  most  cost-effective  manor  over  its 
total  life  cycle. 

4 

Production  &  Development 

Achieve  operational  capability  that 
satisfies  mission  needs. 

3 

System  Development  & 
Demonstration 

Develop  a  system  or  increment  of 
capability;  reduce  integration  and 
manufacturing  risk;  ensure  operational 
supportability;  reduce  logistics  footprint; 
implement  human  systems  integration; 
design  for  producibility;  ensure 
affordability  and  protection  of  critical 
program  information;  and  demonstrate 
system  integration,  interoperability, 
safety,  and  utility. 

2 

Technology  Development 

Reduce  technology  risks  and  determine 
appropriate  set  of  technologies  to 
integrate  into  a  full  system. 

1 

Concept  Refinement 

Refine  initial  concept.  Develop 
system/technology  development 
strategy 

Table  6.  System  Readiness  Levels 
(Source:  Sauser,  p.  7) 


The  SRL  table  (less  SRL  5  which  is  beyond  the  scope  of  this  document)  and  the 
IRL  table  outline  critical  factors  in  the  maturation  state  of  a  system  and  provide  more 
thorough  indicators  to  the  readiness  of  a  system  (or  a  group  of  technologies)  to  transition. 
However,  these  authors  would  argue  the  Stevens  Institute  neglected  the  inclusion  of 
MRLs  in  their  SRL  model.  As  indicated  previously,  in  Section  C,  MRLs  are  a  vital 
assessment  tool  and  would  factor  well  into  the  SRL  model. 
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V.  DARPA  CASE  STUDIES 


A.  BACKGROUND 

1.  DARPA 

DARPA’ s  original  mission,  inspired  by  the  Soviet  Union  beating  the 
United  States  into  space  with  Sputnik,  was  to  prevent  technological 
surprise.  This  mission  has  evolved  over  time.  Today,  DARPA’s  mission  is 
to  prevent  technological  surprise  for  us  and  to  create  technological 
surprise  for  our  adversaries. 

DARPA’s  main  tactic  for  executing  its  strategy  is  to  constantly  search 
worldwide  for  revolutionary  high-payoff  ideas  and  then  sponsor  projects 
bridging  the  gap  between  fundamental  discoveries  and  the  provision  of 
new  military  capabilities.  (DARPA,  Strategic  Mission) 

Mr.  Kevin  Wallace,  one  of  the  authors,  has  experienced  first-hand  the  planning 
and  insight  that  is  required  for  a  Program  Manager  to  fulfill  DARPA’s  strategic  vision. 
PMs  amalgamate  users,  requirements  developers,  stakeholders,  subject  matter  experts 
within  industry  and  academia,  and  transition  partners  to  develop  innovative  ideas  into 
practical  solutions. 

2.  Dr.  Karen  Wood 

Dr.  Wood  joined  DARPA  in  2003,  as  Program  Manager  in  the  Tactical 
Technology  Office.  She  joined  the  Strategic  Technology  Office  in  2006. 

From  2002-2003,  she  supported  SPO  as  an  employee  of  ANSER  working 
on  chemical  sensors  for  WMDs  and  large  space  structures  with  the  ISAT 
program.  While  in  TTO,  she  developed  the  Boomerang  system,  a  rapid 
reaction,  acoustic  gunshot  detection  system,  currently  deployed  in  Iraq  and 
Afghanistan.  She  also  initiated  the  CROSSHAIRS,  RPGnets  and  LASSO 
programs,  along  with  numerous  seedling  efforts.  From  1991  to  2001,  Dr. 

Wood  worked  in  various  positions  at  NASA  Langley  Research  Center  in 
both  the  Advanced  Materials  Processing  and  Nondestructive  Evaluation 
Sciences  Branches.  Her  work  included  materials  characterization  and 
NDE  of  large  space  structures.  (DARPA,  STO) 
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B.  BOOMERANG  PROGRAM 

1.  Background 

The  Boomerang  Program  began  in  late  2003  as  a  result  of  an  immediate  need  by 
U.S.  troops  in  Iraq  for  a  shooter  detection  system.  There  were  several  different  types  of 
detection  systems  already  in  use;  however,  there  was  a  far  less  expensive  system  that 
could  be  more  widely  deployed  and  mounted  on  vehicles,  as  well  as  set  up  at  a  fixed 
location.  The  Boomerang  was  developed  by  DARPA  as  a  Rapid  Reaction  Initiative. 
Under  a  Rapid  Reaction  Initiative,  the  goal  is  to  field  a  solution  within  60-90  days. 

2.  Capability  Gap 

The  genesis  of  the  Boomerang  Program  was  U.S.  Troops  in  Iraq  experiencing  a 
high  number  of  causalities  from  small  anns  fire  while  driving  in  convoys.  The 
adversaries  were  primarily  using  AK-47s  while  hiding  along  roadways  at  distances  of 
less  than  200  meters  and  would  shoot  in  quick  bursts  from  a  concealed  location  (i.e., 
buildings,  walls,  etc.).  Due  to  road  noise  often  times  the  troops  would  not  know  they  had 
been  fired  upon  until  they  reached  their  destination  and  noticed  bullet  holes  in  their 
vehicle,  a  windshield  was  shattered  by  a  bullet,  or  a  casualty  was  incurred.  The 
Boomerang  program  sought  to  remedy  this  problem  by  providing  an  affordable  capability 
of  detecting  and  located  small  anns  fire. 

3.  Boomerang  I 

The  prime  contractor  responsible  for  developing  the  Boomerang  was  BBN 
Technologies.  BBN  was  involved  in  a  previous  DARPA  effort  aimed  at  developing 
counter  sniper  measures  in  the  late  1990s,  during  which  they  developed  Bullet  Ears. 
Bullet  Ears  was  a  stationary  system  using  two  tetrahedral  acoustic  anays  separated  by  50 
meters  that  processed  the  sound  of  a  muzzle  blast  and  the  shockwave  of  the  bullet  to 
locate  the  shooter’s  position  (CNN,  1997).  That  impressive  effort  by  BBN  lead  DARPA 
to  seek  them  out  for  the  Boomerang  Program  (Montiz,  2005).  On  November  17,  2003, 
BBN  was  awarded  a  contract  for  the  development  of  the  Boomerang  system.  This 
contract  included  the  development  and  demonstration  of  a  first  article  prototype  within  30 
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days  after  contract  award  and  included  a  go/no-go  decision  based  on  the  demonstration  to 
produce  50  systems.  It  also  included  six  months  of  research  and  development  covering 
all  engineering  and  software  development  spirals,  six  optional  lots  of  not  to  exceed  190 
systems  per  lot,  and  optional  operations  and  support  of  the  system.  The  initial 
Boomerang  schedule  is  illustrated  in  Figure  8. 
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Figure  8.  Boomerang  Schedule 

The  Initial  Performance  Objectives  of  the  Boomerang  were  as  follows: 

•  Determines  relative  shooter  direction  to  ±  15  degrees  within  1  second  of 
shot 

•  Shot  Detection:  99%  for  bullet  trajectory  within  30  meters  of  sensor  array 

•  Operates  at  up  to  60  mph  on  rough  terrain  or  highways 

•  Weapon  Types:  .20  cal  to  .50  cal  (AK-47  =  .30  cal) 

•  Voice  announcement  and  LED  visual  rosette  display  of  shooter  location 
(‘hours  of  a  clock’) 
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Compact  sensor  array  with  full  azimuth  view 

Demonstrated  performance  in  open  fields  and  urban  environments 

No  false  alarms  due  to  wind  noise,  bumps,  door  slams,  etc. 


Boomerang  Major  Components: 


Figure  9.  HMMWV-mounted  Boomerang  System 
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Figure  10.  Boomerang  Block  Diagram 

Boomerang  I  was  designed,  prototyped,  tested,  and  the  first  lot  of  50  units 
delivered  in  66  days,  only  6  days  over  the  objective  of  60  days.  Out  of  first  lot,  40  units 
were  provided  to  the  Army  and  10  units  were  provided  to  the  Marines.  These  units  were 
deployed  throughout  Iraq.  Users  gathered  data  related  to  system  performance  and  issues 
and  communicated  them  to  BBN  and  DARPA. 

4.  Boomerang  II 

As  a  result  of  areas  for  improvement  identified  by  troops  using  the  Boomerang  I, 
upgrades  were  made  to  the  system  resulting  in  the  second  generation  Boomerang  or 
Boomerang  II. 

The  following  issues  were  identified  from  the  deployed  Boomerang  I  systems: 

•  High  false  alarm  rate 

-  EMI  was  determined  to  be  the  cause. 
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-  Fix:  Shielding  throughout  system;  work  with  EMI  experts  to  mitigate 
problems 

•  Overheated  wiring 

-  Power  conditioners  for  50  units  were  supplied  by  various  vendors  world¬ 
wide.  Several  were  substandard. 

-  Fix:  Removed  power  conditioners  and  use  jumpers  for  initial  systems. 
Current  systems  does  not  require  them 

•  Rubber  hub  isolation  mount  failures 

-  Batch  specific  failures  due  to  rubber  variations.  Drilled  holes  and  used 
doubled-stranded  wire  to  support  hub. 

-  Fix:  Hub  redesign 

•  Speaker  bulky  and  falls  off 

-  Fix:  Speaker  integrated  into  display 

As  a  result,  the  identification  of  these  issues  and  areas  for  improvement 
Boomerang  II’s  Performance  Objectives  were  set  as  follows: 

•  Determines  relative  shooter  direction  to  ±  5  degrees  within  1  second  of  shot 

•  Shot  Detection:  99%  for  bullet  trajectory  within  50  meters  of  sensor  array 

•  Operates  at  up  to  60  mph  on  rough  terrain  or  highways  or  as  a  stationary 
stand-  alone  unit 

•  Weapon  Types:  .20  cal  to  .50  cal  (AK-47  =  .30  cal) 

•  Compact  sensor  array  with  full  azimuth  view 

•  Voice  announcement  and  LED  visual  rosette  display  of  shooter  location 
(‘hours  of  a  clock’) 

•  New  display  shows  range  and  elevation  along  with  10-digit  grid  coordinate  of 
vehicle 

•  No  false  alanns  due  to  wind  noise,  bumps,  door  slams,  etc. 
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•  No  false  alarms  due  to  outgoing  return  fire 

•  System  cost  is  <  $  1  OK 

In  June  2004,  lot  2  was  ordered  consisting  of  50  Boomerang  IIs.  All  of  these 
units  were  provided  to  the  Army  and  were  deployed  throughout  Iraq.  In  order  to  address 
the  identified  issues  with  Boomerang  I  and  meet  the  updated  performance  objectives 
Boomerang  II  improvements  updates  included  a  redesigned  sensor  array/hub,  a 
redesigned  display,  a  simplified  hardware  layout,  and  improved  algorithms.  In  this 
upgraded  configuration  Boomerang  II  met  all  of  its  performance  objectives.  Figure  12 
illustrates  Boomerang  II  improvements. 
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Figure  1 1 .  Boomerang  II  Improvements 
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In  total,  over  500  Boomerang  II  units  were  eventually  ordered  and  fielded. 
DARPA  involvement  in  the  Boomerang  system  officially  in  ended  in  January  2006  when 
its  contract  with  BBN  expired.  User  feedback  on  Boomerang  IIs  deployed  in  Iraq 
eventually  led  to  the  Boomerang  III. 

5.  Conclusion 

Boomerang  is  a  great  example  of  technology  being  quickly  transitioned  out  of  the 
S&T  community  and  into  the  hands  of  the  Warfighter  to  meet  an  immediate  need. 
Boomerang  I  was  designed,  prototyped,  tested,  and  the  first  lot  of  50  units  delivered  in  66 
days.  Once  fielded,  DARPA  and  BBN  worked  closely  with  users  to  identify  areas  of 
improvement  to  the  system  that  resulted  in  Boomerang  II.  After  DARPA’s  involvement 
with  Boomerang  ended,  additional  improvements  to  the  system  were  made  eventually 
resulting  in  the  current  system  configuration,  Boomerang  III.  Boomerang  III  is  a 
commercial  item  available  to  U.S.  military,  law  enforcement,  and  other  authorized 
entities  through  the  GSA  schedule. 
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VI.  SUMMARY  AND  RECOMMENDATIONS 


A.  SUMMARY 

This  Joint  Applied  Project  has  examined  the  current  DoD  technology  transition 
practices  in  the  context  of  organization,  policy,  and  metrics  along  with  a  case  study 
demonstrating  a  very  successful  technology  transition.  A  significant  hurdle  in  technology 
transition  within  DoD  has  been  “bridging  the  gap”  between  the  S&T  and  acquisition 
communities  to  successfully  transition  promising  technologies  from  the  lab  into  the  hands 
of  the  Warfighter.  Recommendations  are  provided  below  on  ways  DoD  can  improve  the 
success  of  its  technology  transition  capabilities. 

B.  RECOMMENDATIONS 
Recommendation  #1 

Recommend  DoD  develop  and  implement  an  alternate  mechanism  to  augment  the 
JCIDS  process  to  rapidly  field  promising  technologies  to  meet  immediate  and  short-term 
Warfighter  needs.  This  would  entail  developing  a  more  streamlined  requirements 
approval  process  for  immediate  and  short-term  needs  where  the  S&T  community  has 
identified  potential  technologies  currently  in  development  that  have  a  high  likelihood  of 
meeting  those  needs  and  being  matured,  developed,  and  fielded.  In  addition,  due  to  the 
difficulty  obtaining  funding  to  transition  technologies  that  are  not  tied  to  a  JROC  vetted 
requirement,  the  alternate  process  would  need  to  address  a  streamlined  funding 
mechanism  to  allow  the  S&T  and  EMD  community  to  rapidly  secure  funding  to  execute 
these  projects. 

Currently  within  the  U.S.  Army,  organizations  are  implementing  a  mechanism 
similar  to  what  the  authors  are  suggesting.  The  Rapid  Equipping  Force  (REF)  was 
established 

to  equip  operational  commanders  with  commercial-off-the-shelf  and 
government-off-the-shelf  solutions  to  increase  effectiveness  and  reduce 
risk,  [and]  insert  future  force  technologies  and  surrogates  to  validate 
concepts  and  speed  capabilities  to  the  Soldiers,  (www.ref.army.mil,  n.p.). 
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The  REF  serves  as  a  solutions  catalyst,  canvassing  the  military,  industry, 

academia,  and  the  science  community  for  existing  and  emerging 

technologies,  (www.ref.army.mil,  n.p.) 

Also  within  the  U.S.  Army,  Prototype  Integration  Facilities  (PIFs)  have  been 
established  to  rapidly  deliver  solutions  to  the  Warfighter.  These  solutions  can  be  organic, 
modified  commercial-off-the-shelf,  or  a  combination  of  the  two.  Mr.  Kevin  Wallace,  one 
of  the  authors,  is  a  Branch  Chief  within  one  of  the  Army  PIFs  and  has  worked  closely 
with  the  REF  on  several  efforts.  He  has  experienced  first-hand  several  successful 
technology  transitions;  however,  the  process  to  bridge  the  gap  between  the  S&T  and 
EMD  communities  remains  to  be  a  somewhat  arduous  task.  The  concept  behind  the  REF 
and  PIFs  is  a  step  in  the  right  direction  and  similar  mechanisms  should  be  proliferated 
across  the  joint  services. 

Recommendation  #2 

Recommend  DoD  implements  policy  requiring  the  industry  proven  practice  of 
using  relationship  managers  in  the  S&T  and  acquisition  communities.  These  relationship 
managers  would  be  located  within  the  military  lab  communities  and  assigned  to  support  a 
portfolio  of  technologies.  They  would  function  as  liaisons  between  the  labs, 
requirements  developers,  industry,  academia,  and  the  acquisition  community.  They 
would  be  responsible  for  interfacing  with  these  communities  to  market  technologies 
within  their  portfolio  and  transition  those  technologies  (when  appropriate)  into  the  hands 
of  the  Warfighter.  As  illustrated  by  the  Boomerang  case  study  DARPA  has  had  success 
using  the  relationship  manager  concept.  The  authors  believe  that  the  relationship 
manager  concept  would  greatly  aid  the  S&T  and  acquisition  communities  in  successfully 
transitioning  technology  if  implemented  throughout  DoD. 

Recommendation  #3 

Recommend  DoD  adopt  a  gated  technology  development  process  consisting  of  a 
strategic  planning  phase,  exploration  phase,  development  phase,  and  a  transition  phase. 
This  process  is  a  common  industry  practice  that  has  proven  to  be  very  successful  in 
transitioning  technology.  In  industry,  successful  transition  begins  with  strong  strategic 
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planning.  During  this  planning  process,  market  needs  and  technologies  the  company  is 
developing  (or  potentially  could  develop  to  meet  those  needs)  are  identified.  If 
recommendation  two  (2)  were  implemented  by  DoD  along  with  recommendation  three 
(3)  the  relationship  managers  would  play  a  key  role  in  this  step  by  working  with  the  users 
to  detennine  their  needs  and  the  acquisition  community  to  satisfy  those  needs.  During 
this  part  in  the  process,  the  following  questions  must  be  answered: 

•  What  is  the  specific  Warfighter  need?  This  entails  open  bi-directional  lines  of 
communication  with  the  end-user  enforcing  the  bubble -up  approach. 

•  How  soon  is  it  needed?  Is  the  need  immediate,  near-term  or  far-tenn,  and  is  the 
timeline  realistic. 

•  Are  there  potential  solutions  that  meet  the  timeframe?  A  thorough  search  of 
potential  applicable  solutions  is  a  necessity.  Are  there  technologies  readily 
available  that  meet  the  user  need,  are  of  sufficient  maturity,  and  can  be 
transitioned  within  the  timeframe  required? 

•  Can  metrics  be  developed  that  lead  to  an  operational  user  evaluation?  TRLs, 
MRLs,  and  SRLs  are  critical  to  objectively  measuring  the  maturity  of  a 
technology.  These  metrics  as  well  as  CTEs,  Key  Perfonnance  Parameters  (KPP), 
and  Measures  of  Effectiveness  (MOE)  help  determine  the  extent  to  which  the 
technology  is  appropriate  for  the  solution  and  guide  the  development  of 
downstream  user  evaluation  criteria.  Additionally,  one  could  establish  technical 
performance  benchmarks  which  can  be  monitored  throughout  the  maturation 
process  to  ensure  the  technology  is  remaining  focused  on  the  solution. 

•  What  is  the  schedule  that  will  drive  the  program?  What  are  the  factors  driving 
the  schedule?  Both  essential  questions  in  detailing  the  path  forward. 

After  strategic  planning  a  gated  process  is  followed  going  through  exploration, 
development,  and  technology  transition  phases,  with  reviews  taking  place  in  between. 
When  a  given  technology  is  ready  for  transition  the  technology  transition  community  and 
the  product  development  community  work  together  to  gradually  shift  management  and 


49 


funding  responsibilities  from  one  to  the  other.  The  authors  believe  that  implementing 
policy  within  DoD  in  line  with  these  practices  would  aid  in  the  future  success  of  DoD’s 
transition  capabilities. 

Recommendation  #4 

Recommend  the  continual  implementation  of  TRLs  and  MRLs  as  maturity 
metrics  with  the  expanded  usage  of  SRLs  to  encompass  the  bigger  picture.  The  ability  to 
discern  the  readiness  of  a  technology  in  an  objective  manner  is  critical  to  the  technology 
transition  process.  A  system  in  development  is  only  as  mature  as  its  least  mature 
component.  TRLs,  MRLs,  and  SRLs  aid  developers  in  identifying  the  areas  of  risk. 
Once  identified,  strategic  plans  can  be  formulated  to  ensure  all  technologies  are  of 
sufficient  maturity  to  transition  to  the  next  phase. 

The  maturation  of  a  technology  is  only  a  portion  of  the  decision  to  transition  to 
the  next  phase  of  development.  Metrics  such  as  MOEs,  Technical  Perfonnance  Measures 
(TPMs),  and  Design-dependent  Parameters  (DDPs)  should  be  established  and  monitored 
to  ensure  the  technology  will  ultimately  fulfill  the  established  need.  MOEs,  TPMs,  DDPs 
and  other  Systems  Engineering  metrics  demarcate  the  technology  as  it  develops  through 
the  TRL,  MRL,  and  SRL  maturity  levels. 

C.  CONCLUSION 

In  conclusion,  this  Joint  Applied  Project  has  examined  the  current  DoD 
technology  transition  practices  in  the  context  of  organization,  policy,  and  metrics.  Within 
DoD  their  continues  to  be  a  gap  between  the  S&T  and  acquisition  communities  that  must 
be  more  successfully  bridged  in  order  to  transition  technologies  out  of  the  lab  and  into  the 
hands  of  the  Warfighter.  In  order  to  do  this,  the  authors  believe  that  an  alternate 
mechanism  to  the  JCIDS  process  should  be  developed  to  rapidly  field  promising 
technologies  to  meet  immediate  and  short-term  Warfighter  needs,  the  use  of  relationship 
managers  within  the  military  lab  communities  should  be  required,  a  gated  technology 
development  process  based  on  common  industry  practice  should  be  adopted,  and  the  use 
of  metrics  measuring  not  only  the  maturity  of  the  technology  but  also  the  extent  to  which 
it  meets  the  users  need  should  be  expanded. 
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